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In July of 2008,1 became Writers’ Center Coordinator and Lead Faculty in English at the Art 
Institute of Pittsburgh. One of the emphases during the interview process was assessment. The 
coordinators I was following (Krystia Nora, and April Sikorski before her) had initiated a 
qualitative assessment of the basic writing course, English 095. It sounded great. It was exciting 
to come into an institution that valued what I did; that was philosophically and theoretically 
aligned with my beliefs. I would inherit a program I fully supported. 

Then I started. 

And I discovered that philosophy and theory are ah very well, but practice is messy. But 
here’s the thing: assessment of writing has to be messy, because writing is messy. It’s not 
objective, numerical, or quantitative. From this messy, subjective process, a great truth arose 
(one I know my mother would disagree with, or disapprove of, or both): Messy things are really 
fun. Splashing in the mud, playing with your food, finger painting. Oh, I imagine you muttering: 
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“Assessment. Messy, yes, but fun? Ha!” I’ve heard assessment called a “perpetual motion 
machine” and it is. It is a wheel that keeps rolling, a circular, cyclical, and sometimes circuitous 
process. But rather than envisioning this circle as the snake swallowing its own tail or a crushing 
wheel of fortune, I like to think of the recycling emblem: three arrows leading the eye around a 
triangle: Reduce, Reuse, Recycle. Unfortunately, assessment is far too often Review, Redo, 
Ignore. Where’s the fun in that? 

We all (I hope) have fun teaching. Brian Huot says, “Because assessment is the direct 
representation of what we value and how we assign that value, it says much about our identities 
as teachers, researchers, and theorists” (11). The qualitative assessment at the Art Institute of 
Pittsburgh has helped us find and express those identities, and that has been valuable as well as 
fun. After all, fellow composition geeks, what is more fun than hanging out with our colleagues 
and talking, debating, arguing and even fighting about writing? The qualitative assessment gives 
us institutionally sanctioned time to do just that. In a world where many overworked department 
members never see each other, we come together several times a quarter to do this work. The 
faculty is extraordinarily dedicated to this project: there is no extra pay and no course reduction 
(although there is talk of doing so soon, as this project requires it). For the moment, there is only 
the occasional pizza as a reward. This group of fabulous professionals is the one part of my 
inheritance for which I am most grateful. 

The Art Institute faculty involved in the initial round of qualitative assessment realized 
early on that they did not know what questions to ask; they only knew that their previous 
methods had not found the answers. A quantitative assessment using the ever-popular grid rubric 
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creates the illusion of mathematical objectivity, but it is just that: an illusion. The rubric guides 
the discussion whether it is appropriate or not. Bob Broad writes of his experience in writing 
assessment: “I notice how all judgments were referred back to the scoring guide or rubric. If a 
criterion did not appear on the rubric, it was discounted as a basis forjudging a text; if a criterion 
did appear on the rubric, it was privileged even if it bore little relevance to the text under 
scrutiny” (25). In other words, when your only tool is a hammer, every problem-or every student 
paper-becomes a nail. Because our faculty values process, creativity, and individual voice, we do 
not wish to pound our students with a hammer. No scoring rubric can reflect our hopes for our 
students. Just as talking in the writers’ center creates better writing, talking among instructors 
creates better writing, reading, and assessment. Donald A. McAndrew and Thomas J. Reigstad 
write, “In tutoring writing, text, reader, and tutor are active” (3). Our discussion of our English 
095 portfolios is an active and interactive process. The students, represented by their texts; the 
readers, the faculty; and now the history of the process, all enter into the conversation. As Walter 
J. Ong states, “Writing from the beginning did not reduce orality but enhanced it” (9). How are 
we to assess writing without that conversation? How are we to reduce writing to numbers on a 
grid? Ironically, when we do the Composition I assessment, for which we still use a rubric-based 
assessment, we find ourselves talking about many of the same issues that the qualitative 
assessment raises. I am now the one who has to crack the whip and bring the faculty back to the 
task at hand, even though I am just as likely as anyone to be the one who strayed in the first 
place. 
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Those issues, in many cases, had not even been recognized before the qualitative 
assessment began. The questions that needed to be asked about our students’ writing emerged 
during the process of discussing the first batch of portfolios. Again, this echoes Broad’s 
experience. He writes, “From the standpoint of qualitative methods, this late blooming is a good 
thing because it means this research question could not have inappropriately guided decisions I 
made in collecting data” (24). Again, the quantitative approach that had been used in the past 
would not have lead us to the central issues that we needed-and still need-to address. 

Early Results 

The biggest impact of this qualitative assessment to date has been on the pedagogy of individual 
instructors and the reunion of the faculty members with each other and with the greater 
composition community. In 2008, faculty members were sponsored at the national convention of 
the Conference on College Composition and Communication to learn all they could about 
assessment, and three of us returned the following year to discuss our work at the Qualitative 
Research Network Forum. In the intervening time, everyone has read and reread both new and 
classic composition essays. We presented at the College English Association Convention in 
Pittsburgh in March, 2009 (conveniently located a few short blocks from our campus) and then 
again at the Writing Program Administrator Conference in July. What was fonnerly a 
disconnected department floating along its own creek (or crick, as we say in Pittsburgh) has now 
joined the mainstream of our profession. In the process, our writing faculty has become 
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energized. Imagine what it takes to get four professors (one pregnant at the time, one with a 
baby, and two with older children) to go to work on a Saturday morning for the love of it. 

The love of the work has created a closer faculty as well. The discussions made us 
excited about teaching again. We were more connected as a department. The timbre of our 
discussions, both formal and infonnal, changed. 

And our teaching has changed. Individual pedagogical changes made in the department 
include a new dedication to student-centeredness, including allowing students the choice of topic 
on major projects; a new approach to and respect for students’ home languages; and a broader, 
more organic definition of research. 

Creating a Sustainable, Longitudinal Qualitative Assessment 

Where do we go from here, and how do we get there? Ah, there’s the rub. First, understand that 
earlier in this paper, when I said “I” I usually meant “them”: the fabulous faculty who started this 
complex assessment project. From now on, however, I am firmly a part of “them” and I hope we 
are firmly now an “us.” I arrived at the part of the process when the first reports were being 
written, and that is when I joined in. So when we submitted our first report, we were trying to 
force a very lumpy round peg into a five-column rectangular hole-the standard assessment report 
within the college. The column entitled “Analysis/Use of Results” seemed hopeless, since the 
first round of assessment lead to more questions than answers. We wrote in our Fall 2007 
English 095 Qualitative Assessment Report that “many of our results, and the attendant analyses, 
are tied to complex issues that do not lend themselves to easy and immediate action” (3). Truer 
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words were never spoken. However, some programs were initiated. We conducted Writing 
Across the Curriculum workshops for faculty, getting faculty to reflect upon their own writing 
processes so that they could complicate their understanding of students’ writing development. 

We had seminars for faculty in working with ESL students and students who learn differently, 
which this assessment pointed up the need for. We called for a continuation of the commitment 
of keeping English095 taught by full-time faculty who have been involved in the assessment 
practice. We also decided to review our meta-compositional reflective piece to give it a higher 
priority in our course design, and we have begun discussion of the various reflective assignments 
and their rubrics in order to ensure that all instructors are giving it similar emphasis. 

This, however, feels like a very small, and far too tidy, step. There are, of course, 
budgetary concerns. Like all administrations, ours is very numbers-oriented, and even more so in 
what everyone calls “the current economic climate.” In a quarter-based academic year, time 
moves very quickly, and it is time to begin the process again even before we’ve had adequate 
time to reflect upon the earlier work. I had to learn to manage the Writers’ Center, develop 
courses, and find my way around the building in a very short time, and, a year later, I still 
struggle with balancing the priorities. Ideally, there would be someone, with some combination 
of additional pay and/or release time, to oversee the process; someone to ensure that the valuable 
work the team has started would not get lost in the all-too frantic shuffle. Perhaps, now that I’m 
settling in to the job, I will find it moves up the priority list, after all. It would be a true tragedy if 
we succumb to the universal frustrations of working within a very slow-moving, traditional 
academic atmosphere and lose the energy and excitement that this process has created. 
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Short of being granted unlimited time and money (a highly unlikely contingency), how 
do we keep this process going? One answer is, given that this process has reconnected us to the 
larger composition community, we plan to continue that connection. We have shared our work at 
three national conferences. We are publishing. We are keeping the conversation alive in our 
professional circle. We have found that we are not alone in what we value, what we hope for, and 
what we need, and that discovery has kept us going. 

We have finished the second round of data collection, have transcriptions, and are in the 
process of coding the new transcripts. We have a clearer sense of what is involved, of what is 
needed, and what is expected of us by our institution. The first study discovered some of the 
questions that need to be asked, and we are beginning to ask them. We are also learning to frame 
our findings in a way that a quantitative world can understand and accept without reducing our 
work to numbers on a grid. 

My hope is that we will find the time, energy, and funding to not only continue a 
qualitative assessment of English 095, but to extend that process to English 1010 as well. We 
have begun the process of tracking the students whose portfolios were used in 095 assessment 
through the course sequence to see if the pedagogical adjustments we have made will bear fruit. 
Any or all of those things will propel us forward. And if all else fails, there’s always our love of 
making a big, slimy, oozing mess. 
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